Summary

Measurement floor for the approved evidence-gated ranking-tuning phase — no ranking behavior change anywhere in this diff. Three gaps closed:

• Canary artifact emission ($0 regeneration path): the eval-canary golden step now passes --json-out and a pinned actions/upload-artifact step uploads .local/eval-canary/ on success and failure (30-day retention = the freshness window). --json-out is decoupled from --json in scripts/eval-retrieval.ts so the artifact file writes independently while stdout keeps the human-readable per-case lines and summary that the tee'd log — and the failure-issue analyzeFailureText step — depend on. Ranking-snapshot regeneration stops costing a paid dispatch.
• Alias-aware snapshot grading: clinicalDocumentAliases/clinicalContentAliases move verbatim from scripts/eval-retrieval.ts into shared scripts/lib/clinical-aliases.ts; the snapshot builder now grades documentMatch/contentMatch through the same sanctioned expansions the live gates use (documentLabelMatchesWithAliases/contentLabelMatchesWithAliases in scripts/lib/ranking-snapshot-builder.ts). Previously alias-satisfied hits (e.g. the EMHS agitation guideline, spelled-out "absolute neutrophil count") graded as misses, feeding the offline tuner mislabeled ground truth. Discriminating tests pin that raw labelMatches fails both cases and the alias-aware matchers pass them.
• Snapshot provenance + freshness: the builder stamps generatedAt (+ optional --source-run-id); validateRankingSnapshot accepts/validates both and relaxes exactly-36 to at-least-36 (floor still rejects truncated artifacts; sourceCaseCount match and per-case candidate minimums unchanged) ahead of the planned fixture expansion. A 30-day freshness test in tests/ranking-tuning.test.ts activates on first regeneration and then blocks silent corpus drift, with the regeneration command in its failure message. New build:ranking-snapshot npm script.

Static hyphen audit of all 36 golden cases' content terms (CIWA-Ar class): no currently-blocked term (canary #52 is 36/36). Residual risk classes documented for the artifact-grounded A-PR-2 pass: punctuation-joined corpus tokens (IM/PO, schizo-affective, post-natal) and inert stem entries (obsess/compuls/hyperactiv/impuls can never match under the whole-token matcher; those cases currently pass via whole-word OR-alternates).

Builder verified end-to-end on a synthetic 36-case artifact: alias grading, provenance stamping, and validator acceptance all confirmed.

Verification

• npm run verify:pr-local — Verification not run locally: hosted pr-required covers build/bundle-scan/RAG-fixture validation; locally ran the full unit suite + the workflow-specific gates below inside the pre-18:00Z merge window
• Targeted vitest 52/52 (tests/ranking-tuning.test.ts, tests/eval-retrieval.test.ts, tests/eval-quality.test.ts), re-run green on the main-synced head
• npm run test — 3019 passed / 1 failed: the known container-only pdf-extraction-budget child-process artifact (baselined on clean main)
• npm run lint + npm run typecheck — clean
• npm run check:github-actions + npm run check:ci-scope — PASS (workflow diff)
• npm run check:production-readiness — expected missing-secret FAILs only (demo-mode container, no live keys)
• npm run eval:retrieval:quality not run: no retrieval/ranking/selection/scoring behavior changed (eval harness scaffolding + workflow only; live gates consume the identical alias maps). Live confirmation is free: tonight's 18:00 UTC scheduled canary runs the unchanged gates and emits the first artifact.

Risk and rollout

• Risk: operational (workflow) + eval-harness scaffolding. Worst credible failures: a malformed upload step breaks the canary job (mitigated: if: always() + if-no-files-found: ignore; pin verified against the repo allowlist), or --json-out decoupling alters eval stdout (mitigated: per-case lines and human summary asserted unchanged by existing eval tests; JSON dump only ever printed under explicit --json). Snapshot-validator relaxation cannot loosen ground truth: ≥36 floor, sourceCaseCount equality, and per-case candidate minimums all still enforced.
• Rollback: plain revert — no cache semantics, no data, no ranking behavior involved.
• Provider or production effects: None from this diff itself. The artifact upload adds no provider calls; it exposes the same content class the tee'd step logs already expose (titles/file names, score telemetry, 220-char previews of the all-public clinical corpus) — no env values or secrets are written to .local/eval-canary/.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use — answer path untouched
• No patient-identifiable document workflow was introduced or expanded — corpus is all-public guidelines; artifact carries the same fields the step logs already print
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy) — workflow env unchanged
• Service-role keys and private document access remain server-only — no new secret exposure; upload path contains eval output only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative — alias tables moved verbatim, zero entries added/removed/reworded
• Deployment classification/TGA SaMD impact: no clinical decision-support behavior changed

Notes

• ADDENDUM 4 Phase A, PR 1 of 2. A-PR-2 (snapshot regeneration + artifact-grounded fixture expansion 36→~46) starts from the first emitted artifact — tonight's scheduled canary if this merges before 18:00 UTC.
• Ledger row appended (docs/branch-review-ledger.md, content commit 81ab969).
• Clinical-governance reviewer verdict: APPROVE-WITH-NITS (P3 only). Independently confirmed: alias tables moved byte-identical; live gate call sites unchanged; artifact payload adds no data class beyond the already-public tee'd step logs; truncation/floor/per-case validator checks intact; freshness gate fail-safe (dormant until first regeneration, malformed values rejected); no new provider calls.
• Tracking for A-PR-2 (from the review): (a) a third, pre-existing and divergent clinicalDocumentAliases in src/lib/eval-document-matching.ts (consumed by src/lib/rag/rag-eval-cases.ts) should be reconciled with the shared module; (b) once the fixture grows past 36, pin the validator floor to the live fixture length instead of the coarse ≥36; (c) after first regeneration stamps generatedAt, consider making it required so a hand-edit cannot silently re-disable the freshness gate.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EXsJcLrbZUXwnBeG91cVo9

Summary by CodeRabbit

• New Features

  • Added a command to build ranking snapshots from evaluation artifacts.
  • Ranking snapshots now include generation timestamps and optional source-run identifiers.
  • Evaluation runs can produce downloadable JSON artifacts while retaining readable console results.
  • Improved clinical document and content matching using recognized aliases.

• Validation

  • Snapshots now support 36 or more cases and validate freshness and provenance metadata.
  • Added safeguards for incomplete or corrupted evaluation artifacts.

• Documentation

  • Updated review records to document the enhanced evaluation and snapshot workflow.